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A3STRA.CT 

The bulk cf research on econcrnlcs ^.education has focused on the 
production funcf|on for teaching economics In colleges and universities. 
This cone ntraticn on higher education 1s notable because the majority of 
students never go to a fcur-year college; they therefore have access to 
formal economics education only 1n 'high school or junior college, 1f at all. 
Moreover, the concentration of the research on teaching In schools (at 
any level) overlooks the fact that most people never have and never will 
take an economics course; for these people, learning economics occurs 
through newspaper columns, magazine articles, television, and other mass 
media. But research on the effectiveness of these mechanisms^ is essentially 
nil. 

Whatever the effectiveness may be of current means of teaching 

t - ' . ' 

economics, and whatever the associated costs--the latter being another area 
that has received scant attention in the 11terature--a key question remains: 
What incen^tives exist for the adoption of efficient instructional approaches? 
Do the Instructors or administrators who qhoose the production techniques 
Ho be used in teaching economics--e.g. , professors versus teaching 
assistants, larger versus smaller class sizes, one textbook versus another — 
have the Incentives to choose efficiently? This is another Important area 
on which the existing research on economic education is silent. 



■Research c ; Economic Education: 

Is It Asking the Right Ouesticns? 

/ 

Burton A. W^lsbrod 
University of W1sconsin-Kad> -i 

1. INTRODUCTION 

The division of responsibility between the two papers this 
session is a fascinating one. One author has agreed to examine the. 
questions that are being asked in the econonics-education literature, and 
the other to examine the answers! 

As is so often the case, however, the underlying assurnption of 
separability does not hold. A research question is not a "good'' or 
"bad"* question independent of the quality cf the answers it is likely 
to generate,. An "exciting" question that is unlikely to yield an 
ans/'er cf substantial Value is not a good question. Research is a 
production process in which something called "useful knowledge" is'che 
output. The inputs to this process include both the specification of 
questions that are important-- i n the sense that the answers would have 
great expected valuG--and the marsf)aling of resources (i.e., the incur-* 
ring of costs) to answer the questions. 

If the costs of answering all research questions were equal, or 
were random with respect to the significance of the question, then the 
separability of the decisions on question specification and on questlo*- 
answering would be justified. What we probably confront, however, 1s j* 
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able tc ansr-i'sr are a'!so t>e "".est cc^t'y ''i.e., cl^ t) . ■":'-e Issue I 

have been asked to deal vr nh--whether research on eccrcric edticaticr. is 
asking the right cue^jticrs-- thus involves implicitly the performance cf 
a bere'^i t-cos t a'^alys'is c'^ v'^c"'*rct se'^^ct^'cr '^'^ f'^e ^rr-^ cf pr^^^^^'^ 
ucctlon research. An evaluaticn is needed of (a) the expected benefits 
•,rore precisely, the probability distribution of benefits conditional on 
answer- of various quality), and (b) the expected costs, of obtaining 
answers of each quality. 

It is ccssibic, ccL-se, t^at a particular research question ray be 
a goer; one the efflciercy sense that the expected costs of researching 
it are less than what the expected benefits would be j[f the resources 
devoted co the research were used as productively as pcssible; yet if the 
resources were not used so productively, it might fail the allocative 
efficiency test. Thus, a question could be potentially efficient to re'-'-^ar -h 
but actually inefficient. In any event, the "best" questions to 
research are those for which the excess of the value of the expected ri • 
swers (benefits) over the expected costs of the research are nax'mi. f^r? 
Deciding which are the "right questions to research implies a benefi' 
cost (efficiency) analysis fo- the prospective project that is es^onti. I/- 
the same as for any otncr res uirce-usi nq project, such as in watc^ Q' 
or rr^anpower training. T'-us . oon ca-'eful scrutiny the imag--' nati ve 
effort by the o-)^ iv: o\ ; ,ession to break a monstrous evaluat'on 
task into two d 'itiici -val uV.: r ; fails to pass the test of separ bi t. 



One basic question 1s, Why study economic education at all? What 
reasons are there tc believe that the subject matter of econooiics Is 
c.-^ficiently special so that the voliicilnous general literature on ^-r. -no 
..nd ::du :at'^c'^> is not applicable to economics?'' I have not se^^n 



-o'i < iior than 150 papers I have surveyed in the Jourr : ic c^^ur 

:\dLcaT2_;c- , and ir. the annual American Economics Associate - ess^ '^n'E 

2 

)n - ' ducation. (There are', of course, papers on econc . : ec: icr r- 
pu ^whe'^e, bjt my survey does not extend much beyond ^ ;e twa 

A sources.) My point is simple: Is there not a r.ant^'a 
V- dee^ri, perhaps not a presumption—that researcher jtudyin 
ed tic are "rediscovering the wheel," posing and answering 
■i;.'^ :^ ir th 'av been answered previously in the more general researcl on 
r or ' For •e>ample, is it not likely that the effect on "lear'^^^"*g" c t\ 

sd H^iS siZ' , or of the use of teaching assistants rathe^^ than more 



experienced prcl^esscrs, cr 1 rdi vidua' 'y sel^-paced arprcacries rather 
than a tradlticral ?-rifcrr» Irstructcr-caced apprcscr, is similar -:t all 
subjects? I do not assert that the answer is obvious and affirmative. 
I only question whether it Is a "high priority" research' matter to devote 
>ubstant1al resc-ces to ge-e-al ;.cst1crs cf teaching techniques, questions 
that are not specific to the teaching and learning of economics and that 
have been studied extensively In other subject rratter contextr>. It may well 
be true that, as one c':nnoni1st at an AEA'se- ion o.n •conomi , uic ticn 
recently put It, '^Educational production fui . ^Morv: ar e a bet cs 
interesting as these for hybrid corn" (Allijc - .2:- , ■ - 

it wot <i not follow that production func^i?' > ,r v 
eff'^'c r top.fcs for econorr^ists ' researc 



3. " ' '^'nniCTION FUNCTION FOR ECONOMIC .j!;' .\T N 

t ::earch has been undertaken 1r ^:^concntc:s eojc. ^ Mont o- 

r^^- 'Exploring socie portion of he p-oduct1^nn f u ic n ^r ecnmorr, . 

Of the 159 papers surveyed. I count lfi2— e : ri )y tw-th * 
dr - ither with how to define "and measure outr;jts {? pr -mrs), cr w :h the 

effect . i output of various alternative. Inputs (73 paper ; This production'^ 

:^functior; orientation is consistent, however, with the Jc :r-c of Economic 

Education 's ( JEE ) goal as stated inside thz^ front cover: i pron- !c the 
teaching and learning of economics in colleges, junior ( iTp and hig^ 
schools by sharing knowledge of economic education/' 
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studied, ^nd according to the level of schcoHng. Since each p'aper was 
counted only once, while sor;e papers touched on r.ore than one input or 
school level, the ta^le d-- inconrlete por'Tayal o"' the r^rsear-"^ foci. 

I have classic ^-^i-pendent varlal r in :he prc^duct -im function 

as capital, labor, si;d«^ v i ^^nselves, cour^ e . ntent, ard 1ns ..r*uction^l 
nethcds (ways of conb'i ig f nf: ,:..:). An I'^prf^n ; - e variety c ^ va'^iabTes 

can^' judge whethe- ^q^^ inputs th.at h.^^ve 
ont ; . are \.'orth" , lng--for e.v the 
c y that the c' :5 held (b..i : Kirus, 



have been researched, 
received little or no 
output effects of :he 
1973), the'color-'Cf 



ocTi wa i i s , or 



..'ating arrsr?) tiin*:- nts 



Interaction Effect: 



;iu: r ^^rious OMissic- t:he lack c e/a^jnation 
dmr.n;j ' ..ut variables seems likely, -or ex- 



What i^ proba* 
of interaction eff .- 
ample, that a partv. jl r -y^e of textbook (iipu' lA) when used by grad- 
uate teaching assis^-ir^s M ii: ) will be more effective for low-ability 



s tudents ( 1 1 : ) *thari 



be for high-ab'lity students; Similarly, 



games and sirulatlorrr in:M.* VA) nay be "di ffer -ntially effective depen>iing 
on whether, instructor; 'IIA )r ^jraduate assis i^nts (IIB) are used and 
depending cn the st^-if:'. vvitai level of motivation (III). 

Li mited Sc ope 

- Another striking aspect of Table 1 is the overwhelming einphasis on 
teaching at the college level (77 percent of the papers). The JEE goal , 



i) 



Level of School 



r- np ^.f " 
Ca pi tal 

Textbooks 2 2 ' 4 ' 

Ccnputers 5 -5 

Televlsicn, slides, etc. 1 i .17 

Ir^str. Iters ' ^ ■ 

Gradjate Ass i ^ t::' :^ 5 = 

' r; r . , 1 t r ^' r ' *; T 



S tudents (<3bil1ty, rot:- 

vation, f(ir]ny bac^^ro-r^ii-f 4 5 ^ 

qther^ students ) ' - > -/ 



Coarse _Contprrt ^ 

(subject matter) 2 ? 7 ^ 11 (14"^) 

J ^ ructio nal Methods 
(way's of combining inputs) 

Games and Singulations 8 
Learning contracts, self- 
pdced instruction ^^nd 

pro'jrann^d learning , 11 ]] 

Lectures 1 1 

Course evaluations 4 ' 4 

Length of course • 1 ] 

Class size * ] f 

26 ( 330 

Total 12 5 ~0 rj '79~ 

(^5'] (6 J . (77M • ■ (1-) {loo;) 



education 1n junior collece: are Weldenear and Dodson (1972) and Lewis, 
Wentworth and Orvis (1973). For a precollege focus, see Pels (1977) 
and Duff (1971). It ray or nay not be true that the production furxtion 
findings for the college population apply also to the junior colleges and 
•-^ijh scbccls; t-^e issue reri ts attention. Students' ability and motivation 
levels (as well as the va-iar.ces in those levels) vary across the schooling 
levels; thus, the i nteractlcr; of these student characteristics v^th 
other, conventional inputs will produce, T hypothesize, different output 
effects depending on the level of school. 

The narrow scope of teaching settings on which research has been 
published is also evident frorr the dearth of attention to the production 
function for teaching economics either in graduate schools (see, however, 
Hansen (1971) and Decker (1973) for models predicting succ9ss in graduate 
econoTiic studies) or in nonschool settings such as in the home via television 
•see Coleman. 1963) or via popular journalism (magazines and newspapers). 
How "effective," for example, have been the syndicated newspaper 
columns of writers such as Sylvia Porter; the N ewswee k columns by Kilton 
Friedman and Paul Samuelson; the articles in magazines such as Challenge 
or Public Interest, or in daily newspapers? How effecti ve--and for whom— 
are the efforts of private firms to provide "economic educatior.*' via 



jndnswered--indee'^ , unas.<6c-c jest 1 crs . Yer, the vast ^ajcrity or people 
have not tai'en and nsver will take a foraal eccnorr.ics course In an/ school, 
and they will be exposed tc econoTilcs only through such Inforr.al redla. Thus, 
the Drcductlcn fjnction for learning ccononi':s outside tradi tion.=i'^ sc-^'oo'^s 
seefTs to warrant substantial exploration- -assuTning , of course, that 
ecpnofnl^s i'; worth the opportunity cost of learning it. The otnisrion o- 
nonschool teaching and learning of economics from the J [IE staterrcnt of 
policy^ is unfortunate. ^ 

Distri butlona l Effec ts 

I turn next to a related aspect of the production function work- the 
distributional effects of alternative course contents, input conblnatlon:; , "'^ 
and Instruction^} materials. These have been studied to some extent (for 
example, Attiyeh and Lurnsden, 1972; Hansen, Kelley and Weisbrod, 1970; 
Thompson, 1970); yet, given the evidence from the general literature on 
education thac a giver approach is likely to have substantially different 
effects on different "types" of students, this dimension seem?; to deserve 
more scrutiny. Whatever th( mean differential may be between the output 
effects of different Inputs, examination of the variance about the mean may 
disclose systematic differences among students according to characteristics 
that are discernible at the outset of a course. 



IS 



4. OUTPUTS . * ^ • 

Goals of Economic Education 

i . I >^ J 

The body of research prescinted 1n Table 1 focuses on the productivity of 
various inputs ; the dependent variabl e--output~-i s generally taken as 
()iven,ty cally in the form of some test score. There is, hoWfever, sub- 
stantia"^ otht?r 1 i terature— not in the input-output framework--di ^cussing 
the normative question of how out put ought to be defined and measured. 
There are papers that discuss the "usefulness" of a specific output meas- 
ure, particularly the Test of Understanding in College Economics (TUCE) (for 
example, see LeWls and Dahl , 1971; and Fels, 1977), Other concepts and measures 
of outputs on which papers have been published Include changes In student 
political attitudes (Sott and Rothman, 1975) the students' own judgment of 
effectiveness (Kelley, 1972); learning "radical" economics (Edwards and 
MacEwen,^ 1970; Gurldy, 197E); and developing problem-solving abilities 
(Pels,- 1973), In addition, the durability or permanence of the effects, as . 
distinguished from measures of effectiveness obtained at. completion of 
the course, has recelved^a, 11 ttle attention (see Saunders, 1970; and 
Saunders and Bach, 1970). 

OveralT, however, the question of what economics eduratlon ought to be 
aiming at---that Is, which outputs should be produced-r1s a question that 
has received little riigorous analysis- The question of what kind 
or kinds of . ^'economic education" to produce 1s a difficult one. 
Should It be Ideologically oriented? ShouTd it provide whatever, "buyers" 
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v^ant7 Who are the buyers-'-parents? taxpayers? students? Our customary 
consumer-sovereignty model appears to be of Hmited guidance here, given 
widespread consumer Ignorance of the Importance of economic knowledge, and 
given the external benefits from having a population that is more sophis- 
ticated in its understanding of economic processes. 'In economics educa- 
tion, as in many other "professional" markets, buyers are poorly informed 
regarding product quality. Even if buyers know their objectives, they may 
know little about the effectiveness of particular activities in achieving 
those objectives. 

My references to "consumer ignorance" and to "external benefits", how-- 
ever, are scarcely more than assertions. I have seen little research that 
rigorously confronts the question of whether 'there is a market failure in 
the economic education market, with too few people studying too little eco- 
nomics or studying the "wrong" economics. The published research either 
asserts that more economics is "good"--and presumably is better than some 
unspecified alternative uses of student time and other resources--or else 
the research asks the narrower production-function question of how effective 
one type of input js compared to another, without aski ng Whether the output 
Is worth-.produclng. In volume 1 of the JEE Stigler (1970, p. 78) did pose the 
question, "Why should people be economically literate, rather than musically 
literate, or historically literate, or chejnically literate?". I must resist 
the .temptation to discuss his answer~-except to note that musicians, histor- 

\ 

ians and chemists may see things differently. 

Effectiveness versus Allocatlve Efficiency • ' 

The domination of a production function emphasis In economic educa- 
tion research has, in short, obscured the related Issue of the allocatlve 

,. ' 14 



efficiency of alternative input combinatibns . Many 'papers have examined 
the effectiveness (productivity) of var »ous inputs, but rdire]y have the 
relative cj}5t_s of the inputs been juxtaposed to the relative effective- 
ness, nor have the measures of effectiveness been translated into values 
of benefits. These questions have seemingly been overlooked or, at least, 
si ighted. 

I find it surprising that among tite (admittedly small number of) 
papers confronting the question of how to define the output or outputs 
of economic education, there has been so little attention to labor 
market effects in general, and earnings effects in particular. The 
contrast between the economic education literature and economics of 
education literature is dramatic. The Vatter has concentrated, typically 
within a human capital theoretic framework, on the relationship between 
education (meaning schooling) and earnings, virtually disregarding the 
process through which educational inputs produce the outputs that have 
value in the labor market. Another way of saying this is that the eco- 
nomics of education literature has viewed earnings as the value of outputs. 
Meanwhile, the economic education literature has concentrated heavily on 
the process ^of converting inputs into outputs in nonpecuniary forms, vir- 
tually disregarding the valuation of outputs. 

One might have predicted a priori that the economic education litera- 
ture would have included numerous Efforts to assess the labor market value 
of economics training either directly or indirectly through its effect on, 

say, the probability of admission to law school. Why the economic education 

S 

literature and the economics of education literature have been so divergent. 
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and whether either, or both, or neither has pursued an "optimal" path are 
questions which I raise here, but will not pursue far, 

Lifetime Effects 

One further observation, however. The human capital framework, with- 
in, which much of the economics of education literature has been cast, has 
focused research attention on the investment aspect of schooling (typically 
in broad units such as high school or college, though sometimes in .narrower 
program units such as vocational training or even Ph.D. training in economics). 
The investment emphasis Implies a 1 i fetime perspective on the Q.utputs of 
schooling. By shar^ contrast, the economic education. 1 i terature has con- 
centrated overwhelmingly on the immediate outputs, those measured at the 
completion of the course.' As pointed .out above^ there have been a few note- 
wopfhy exceptions, in which the durability of outputs has been considered, 
though even- these have Involved a horizon of only a few years or so (Saunders, 
1970; Saunders and Bach, 1970). It fray well be exceedingly difficult to measure^ 
lifetime effects of exposure to economics, and this may explain the lack of 
attention to this question In the literature. (This would Illustrate the 
Interrelatedness of the "do-ability" of research and the formulatfon of • 
research questions.) But the fact remains that little effort has been 

devoted to the measurement of llfeti/ne effects • 

✓ 

5. INCENTIVE STRUCTURES . « 

Another underresearched area Is the nature of incentive structures 
facing teachers and administrators. Assume that (1) the production fu/iction 




IG- 




13 

research disclosed that certain Inputs are more ef fective than others ;^ 
(2) consensus was reached on appropriate i '^es of outputs (I.e., 
effectiveness), and (3) outputs and Inputs were valued and showed positive 
net benefits from a change In current teaching practices. Would the changes 
occur? Are there Incentives sufficient to encourage changes that are 
efficient (granting that such changes can be Identified v^ith reasonable 
confidence)? 

These .questions , It might be argued, transcend ^u, nic education.'^ 
It would seem, however, that the responsiveness of -acrr^-^r and admini- 
strators of economic education programs may or may K" the same as 
for those In noneconomlcs areas; at least this hypotK. ; cannot be ruled 
out, any more than carj^the hypothesis that variation I i 'ass size, or 1n 
the effectiveness of teaching assistants or the use oi r revision instruc- 
tion differs as between economics and other subject area^. 

The nature of incentives confronting teachers — of economics or of any- 
thing else, and at various levels of school 1ng--has received scant attention. 
There are possible Incentives for instructors (a) to learn which changes are 
efficient, and (b) to make those changes. (On the latter point, however, 
studies of salary determination at universities [e.g., Siegfried and White, 
1973; Koch and Chizmar, 1973] have shed some light on the financial returns to 
scholarly research, teaching, and other uses of faculty time.) It is argu- 
able that little is to be gained from research on how to "improve" teaching 
if the incentive s to ddopt improved methods are weak. It is also arguable, 
on the Ouher hand, that incentives are weaker than they might be because 
there is so little agreement as to what constitutes efficient teaching; this, 
after all, involves the specification of goals in operational terms and the 
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adoption of v 'ghts for the multiple ( ""hat surely exist. Thus, 

understanding and weights Is one part n research agenda for 

efficient Innovation in education. 

In any analysis of incentives in education the relationship between 
private costs and social costs (or returns) Is likely to be crucial. As an 
Illustration, consider the case of an economics instructor who is free (although 
many are not) to select a undergraduate textbook, and that a nfew textbook 
appears on the market. may well be little incentive ^^^^ )c1al , pro- 

fessional^ or any other ead the new textbook careful,/ - to de- 

termine whether it is superior to the one beingused; th s, nowever, is not 

my principal point. What if the ir^structor knew--costl e sly and with cer- 

/ 

tainty--that the new book was "more effective'* for all ct his students; what 
would be th6 private an J social costs and benefits of a :Tr :)ting the new book? 

- \ ■ 

Of course, "more effective" need not imply *'more efficisrt." 

From the students viewpoint, the new book would presumably be preferred 
it if were more effective. Such a preference, in turn, embodies two deeper 
assumptions: (a) the similarity^of student goals and of faculty goals for 
students, and (b) the absence of higher, costs (time, effect, money) for using 
'the new book that offset the benefits of increased learning. 

Note, however, that while the student must incur the cost of reading 
whatever textbook is chosen--so that this is essentially a fixed cost--the 
vacuity person bears an increased real social cost of changing, since he or 
she has lecture ^ tes keyed to a textbook that has already been read. With 
the benefits of* change accruing to students while the costs are borne by 
faculty, the likelihood of market failure is substantial. 
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The market failure would disappear, however, if the instructor in- 
ternalized the students* benefits. This might appear to be the case if 
the instructor acted as an idealised "professional "---that is, acted as the 
consumer • s. agent for maximizing the consumer's (student's) utility. Educa- 
tion is an example of a commodity—1 ike medical care ano legal represe^Mta- 
tion--in which consumers are aware of their iriabiliti to judge quality and 
so they place trust in the professional to act in their best interest. 
Even if the instructor were to behave, however, so as to maximize not his 
or her own utility but that of students [or parents, or taxpayers); it 
would not follow that efficient resource allocation would result\ The 
reason is t ,at the cost of switching textbooks (or, in general, of chang- 
ing anything in the teaching process) is a reaf cost; if it were to tie 
disregarded--as would be the case if the instructor w^re to act so^as to 
maximize the consumer's utility--the result would be excessti^e change. 

The market failure would also disappear if the reward structure were 

such that the instructor's pay were an appropriate function of the "value 

added." Then, if students learned more from the new text, the Instructor-- 

acting in sel f-1nterest--woul d weigh the costs of changing books 

against the benefit,^ and would choose accordingly. Ideally, the re- 
* 

wards would be commensurate, with the student benefits and S!C>--assuming 
away real external effects and other market 1mperfect1ons--the instructor 
would be confronted with the real costs and benefits of change. .The prob- 
lems of developing such a reward system are\doubtl ess gr^t. It does not 
follow, however, that they are not researchabie. 

\ 

I* 
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These remarks have been abstract; meat must be put on the analytic 
bones. I hope that the next time the economic education lit' *at e 1. 
veyed there w |Um "S exploring incentlvt s l\^r novat' 

and efficiency- v;s t ve ai d normative dimensions. 



6. CONCLUDING r S 
\ 

It is all ic mple 
researchers to ta 'e» ar 
reiterating »r ^ claim mad( 
research que^ tic s is, i 
costs, of cofrpai ing th 
ing them. If t^e costG . 
to research questions the 



ind questions that one would like th " 
ve done here. Thus, I should clos l 
the outset that the selection of op' hi > 



iciple, a matter of weighing benefi s 
. jf having answers to the costs of 
lu, rficiently high, it would be ineff 
eem important. Some of the questiOf . 
which I have pointed probab / fail such a benefit-cost efficiency : 
and so have received, quilt^ wisely, little research atten*:ion: otl^ 
questions, however inay pa it--at least for somo rosea 
merit rore s Once mc! e we can concludj that "more ,:arch' i: 

neoded." 
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NOTES 



»1 to *^ay that there Is nothing special about- the t(a :h1n() 
of li s. Economists typically believe, for example^ that :n ople 

hav.' imis information and biases concerning economics than abcut 

..Dject matters. (Mark Schleslnger pointed this out to me.) Even 
.T i is true (see Boulding, 1975) the question would renain whether 
^ -A- .:es devoted to teaching economics shoul3 be deployed different ''y 
nub 'ects . 



ellent survey of research on educational product: 
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